The present invention relates generally to a video surveillance system using background subtraction to identify foreground abnormalities. Particularly, the present invention relates to the learning of a background model for a scene of interest that is robust to drastic lighting changes.
For video surveillance systems using background subtraction to identify foreground abnormalities, the computation of a proper background model over time plays an important role that dominates the system performance. For applications of long-term video surveillance, the computation of an up-to-date background model is needed to prevent incorrect detections of background scene changes, e.g., lighting variations from dawn to night, as foreground alarms. To this end, periodic updating of background model estimates is often adopted by many surveillance systems to learn background scene changes. We regard the approach of periodic background model updating as background model learning.
In last decades, many approaches addressing the problem of background model learning, e.g., C. Stauffer and W. Grimson, “Adaptive background mixture models for real-time tracking,” in Proc. IEEE Conf. CVPR, vol. 2, pp. 246-252, 1999, have been proposed. For most background model learning systems, the frequencies of background model updating (for all pixel locations), also known as the model learning rates, have large effects on the system stability. (The definition of model learning rate is inherited from C. Stauffer and W. Grimson, “Adaptive background mixture models for real-time tracking,” in Proc. IEEE Conf. CVPR, vol. 2, pp. 246-252, 1999.) Through periodic background model updating, various background scene changes, including lighting variations, resting objects, etc., will be adapted into a background model. The more frequently a background model is updated; the more scene variations are adapted into the learnt model, which results in a surveillance system (based on background subtraction) being more robust to background interferences and less sensitive to foreground abnormalities. As a result, applying high learning rates to a background model learning system will result in insensitive foreground detections to intruders in slow motion, because the system tends to incorporate slowly-moving intruders into its background model via frequent model update.
As most background model learning systems need to operate at moderate, but not high, learning rates to fit general surveillance conditions, gradual (and perhaps quick) lighting variations may thus be captured by the background models computed from these systems. However, for some over-quick, also regarded as drastic, lighting changes induced by, e.g., sudden sunshine varying, such systems may then become inefficient. Many false detections of foreground regions resulted from drastic lighting changes will hence be generated. The lack of efficient and effective ways in handling drastic lighting changes for general background model learning approaches motivates this invention.
To enhance the background model adaptation to lighting changes for general background model learning systems, a new system design is proposed by acquiring a lighting change processing unit as a post-processing module for these systems to revise their foreground region detection results under drastic lighting changes and to guide the dynamic adjustments of their model learning rates. With the proposed post-processing module, the capability of model adaptation to drastic lighting changes for general background model learning systems can be largely improved, without the need of manual tuning of model learning rates.
FIGS. 1A-1F give two examples of quick and over-quick (drastic) lighting changes. The image sequence shown in FIGS. 1A-1C records a laboratory with a CRT monitor displaying rolling interfaces and with quick lighting changes, wherein FIGS. 1A-1C show two images of image sequence IA recorded at 20 fps experiencing quick lighting changes. In this image sequence, it takes about three seconds to increase the average intensity by ˜20%. This kind of quick variation in image brightness is able to be adapted by common background model learning approaches, e.g., C. Stauffer and W. Grimson, “Adaptive background mixture models for real-time tracking,” in Proc. IEEE Conf. CVPR, vol. 2, pp. 246-252, 1999, by applying a higher learning rate (than the default value suggested in the paper). See FIGS. 2A and 2B for the simulated foreground region detection results for image frame ItA by applying Stauffer and Grimson's method. On the other hand, for the case of over-quick lighting changes shown in FIGS. 1D-1E, which show two images of image sequence IB recorded at 15 fps experiencing over-quick (drastic) lighting changes, similar increases of image intensity are observed in less than one second for an outdoor environment. As shown in FIGS. 2C and 2D, which shows foreground region detection results for the image frame ItB in over-quick lighting, many false detection results in foreground are received under such a condition by using the same approach with an identical learning rate setting. Instead, by applying the proposed background model learning system according to one embodiment of this invention, most false positives of foreground region detection are eliminated, as shown in FIGS. 2E and 2F.